1. Field of the Invention
The present invention relates to a method and apparatus for transforming a moving picture coded by a coding system into another moving picture coded by another coding system.
2. Description of the Related Art
With the progress of multimedia techniques for handling voice, image and other data collectively, conventional information media such as a newspaper, magazine, television, radio, and telephone, which are means for conveying information to people, become subjects for the multimedia.
Generally, the multimedia are considered to show not only characters but also other information such as figures, voices, or particularly images, associated with characters at the same time. In order to make the above-mentioned conventional information media the subjects for the multimedia, representing the conventional information in digital format is the essential condition.
However, the estimation of the code amount of each of the conventional information media in digital format results in 64 kb per second for voices (telephone quality) and equal to or more than 100 Mb per second for moving pictures (current television reception quality), while the code amount for a character is 1 to 2 bytes. Therefore, it is not realistic to handle such a massive code amount in digital format without using any transformation.
For example, a video teleconferencing system is already put to practical use with an Integrated Services Digital Networks (ISDN) with a transmission rate of 64 kbps to 1.5 Mbps, it is however impossible to transmit images for a television or camera directly without the transformation with the ISDN.
A data compression technique is consequently required. For example, the video teleconferencing system uses moving picture compression techniques of the H.261 standard and H.263 standard that are international-standardized by the ITU-T (Telecommunication Standardization Sector of International Telecommunication Union).
Further, according to the information compression technique of the MPEG standard, ordinary television broadcast moving picture is compressed to 2 to 15 Mbps.
Herein, the MPEG (Moving Picture Experts Group) means the operation group (ISO/IEC JTC1/SC29/WG11) working for the standardization of speech and moving picture coding in the ISO (International Organization for Standardization), and also means the international standardization of the data compression that this group defines, at the same time. Currently, the MPEG has standardized MPEG-4 enabling the coding and processing by an object basis to achieve the new function needed for the multimedia technique.
A plurality of moving picture coding standards are thus to be used, resulting in the problem to be solved that coded data (hereinafter referred to as bit-stream) in such standards is not compatible with each other.
For example, although a decoder based on MPEG-4 is specified to be capable of decoding H.263 bit-stream, there is generally no compatibility of bit-stream between coding systems of H.261, H.263, MPEG-2 and MPEG-4 (in other words, it is not possible to use a coder and decoder based on different coding systems as a pair).
This condition requires an apparatus that transforms bit-streams based on the different coding systems (hereinafter referred to as transcoder).
FIG. 1 illustrates a conceptual diagram of the transcoder. In FIG. 1, different coding systems are A and B, decoder 101 with a system A and coder 103 with a system B are connected through a frame memory and image transformer 102.
For example, it is assumed that the system A is MPEG-2 (picture size: 704xc3x97480 pixels), and that the system B is H.263 (picture size: 360xc3x97288 pixels). Although the picture sizes thereof are different from each other, it is possible to perform the transformation by once transforming the bit-stream to an image, and then coding the image again.
In addition, the H.261, H.263, MPEG-2 and MPEG-4 coding systems use Motion-Compensated Discrete Cosine Transformation coding system (hereinafter referred to as MC-DCT coding) as a common method. In the MC-DCT coding system, an image is divided into blocks each of which is called a micro block comprised of 16xc3x9716 pixels, a shifted amount (motion vector) that minimizes a differential between successive images (frames) is calculated, and the differential is subjected to Discrete Cosine Transform (DCT) coding.
The processing to obtain the motion vector that minimizes the differential between the frames to obtain a differential image from which the redundancy is removed is called motion compensation.
DCT is used to remove spatial redundancy left in the motion-compensated image (differential image).
In addition, a first frame is coded using only DCT because the differential coding with another frame is not performed. Such a frame is called I picture, and other general picture to be subjected to MC-DCT coding is called P picture.
If the MC-DCT coding system as described above is used in the coding systems A and B, it is possible in decoding in FIG. 1 to obtain secondary information (hereinafter referred to as side information) generated in the MC-DCT coding such as moved information and DCT coefficient quantization, and it is expected that the use of the side information improves the computation efficiency and coding efficiency (image quality and coded content) in recoding.
A conventional example of the transcoder using the side information is a moving picture coding transformation apparatus disclosed in Japan Unexamined Patent Publication HEI10-271494.
FIG. 2 illustrates the concept of the apparatus.
When it is assumed that the systems A and B in the transcoder illustrated in FIG. 1 are respectively MPEG-2 and H.263, a configuration of 101 to 103 in FIG. 1 corresponds to that of 202 to 204 in FIG. 2.
In the conventional example, since the two coding systems have the MC-DCT coding system as the common method, it is possible to omit motion vector searching in H.263 coding by executing scaling (transforming an absolute value by multiplying a ratio of picture sizes) on the motion vector obtained in MPEG-2 decoding.
As illustrated, for example, in FIG. 2, the input image (MPEG 2) and the output image (H.263) have different sizes. Thus, a motion vector scaling block 205 is provided and performs scaling of a motion vector of an input stream (MPEG 2) to transform the same to a size appropriate for the size of the output image and provides the result (H.263) to the coder 204.
In the conventional example illustrated in FIG. 2, with the motion vector adopted as the side information, the computation efficiency in recoding is improved using the side information obtained in MPEG-2 decoding.
In composing a practical transcoder, however, there are many subjects to be examined other than the reuse of motion vector.
In the practical coder, a process called rate control is inevitable to adjust the code amount of bit-streams output therefrom.
Generally, an output buffer is installed at an output side to absorb a variation of generated code amount, and corresponding to the buffering data in the output buffer (hereinafter referred to as buffer loaded code amount), the coded content is adjusted.
In the MC-DCT coding system, the coded content adjustment is executed by setting the quantization accuracy for the DCT coefficient to variable (fine accuracy improves the image quality and increases the generated code amount, while rough accuracy results in the opposite effects).
FIG. 3 illustrates a conceptual diagram showing subjects for the transcoder.
An output terminal of output buffer 304 is connected to a communication path or storage medium. When it is assumed that the communication path has a fixed data rate, the bit-stream is output from the output buffer with the constant rate. Coder 303 with the system B adjusts the generated code amount so that data in buffer 304 is not consumed (underflow) or not excessive (overflow). Meanwhile, in order to improve the recoded image quality, it is necessary to use the side information concerning coding procedure such as quantization and coding mode. Generally speaking, when a coder with the system B is operated independently of the coding procedure of the system A, an image quality loss generated by the system B is added to the image quality loss generated by the system A.
For example, it is assumed that the bit-stream of the system A is coded at a rate of 10 frames/sec. When the reproduced result is recoded by the system B independently at a rate of 15 frames/sec, a frame with the system B of which the coding time accords that with the system A is one frame in three frames. In other words, the decoding intervals of other two frames do not accord with the original moving picture. Therefore, despite the frame rate being increased, the decoding result by the system B may not maintain, or even decrease the image quality of the system A.
The same phenomenon occurs in the quantization of DCT coefficient. It is assumed that the quantization in the system A is executed by rounding with xc2xd. In order to prevent the generation of accumulated errors due to requantization, it is the most preferable that the quantization in the system B is also set to xc2xd, to obtain a symmetry coded content. The frame rate conversion, requantization of DCT coefficient and picture size conversion are processing all to execute xe2x80x9crequantizationxe2x80x9d, in the broad sense, of moving pictures. For the requantization, it is preferable to operate coder 303 with the system B depending on decoder 301 with the system A in FIG. 3.
A decompressed image output from system A decoder 301 is temporarily stored in frame memory/image transformer 302 prior to being output to system B coder 303. In addition, when an input stream and an output stream have different image sizes, the frame memory/image transformer 302 perform image size scaling on the output stream.
As described above, in order to execute the rate control, it is preferable to operate coder 303 with the system B independently of decoder 301 to enable the control of requantization, while observing the loaded content in output buffer 304. On the other hand, in order to reduce image quality deterioration by the broad-sense requantization, it is preferable to operate coder 303 with the system B depending on decoder 301 with the system A. However, the conventional techniques concerning the transcoder is developed from a point of view to improve the computation efficiency by using the side information, for example, recoding by motion vector scaling, and a technique is not disclosed that achieves both rate control and suppression of image quality deterioration due to requantization.
In view of the forgoing, an object of the present invention is to provide a method and apparatus for performing a rate control while suppressing image quality deterioration in transforming a moving picture coding system.
A first aspect of the present invention is a method and apparatus for transforming, sequentially from a head of data, a first data sequence generated by a first moving picture coding system comprised of an intra-frame coding mode and inter-frame coding mode to a second data sequence generated by a second moving picture coding system comprised of the intra-frame coding mode and inter-frame coding mode, and the method and apparatus
1. read data of a frame from the first data sequence, and further preread a coding mode of a successive frame; and
2. decides whether the present frame should be coding or not for decrease a generated code amount, corresponding to a generated code amount of the second data sequence transformed to the second moving picture coding system in the case where the preread coding mode of the successive frame is the intra-frame coding mode. Then the data coded by the intra-frame coding mode existing in the first data sequence is preferentially transformed to the second data sequence.
According to such method and apparatus, it is possible to perform stable coding which does not cause the overflow in coding an xe2x80x98Ixe2x80x99 frame, by prereading the coding mode of the successive frame, and controlling the coded content of the present frame corresponding the coding mode of the successive frame and the coded content generated in the system transformation.
A second aspect of the present invention is a method and apparatus for transforming a first data sequence generated by a first moving picture coding system to a second data sequence generated by a second moving picture coding system, while transforming a frame or a picture portion obtained by dividing a frame on a coding-unit basis sequentially from a head of data, and the apparatus and method
1. record a code amount of the coding-unit in reading data of the coding-unit from the first data sequence, and
2. set a target value of the generated code amount of the second data sequence at a value obtained by multiplying the recorded code amount by a predetermined factor in transforming the data of the coding-unit to that of the second moving picture coding system, and then control a rate so that a ratio of code amounts, on a coding-unit basis, of data composing said first data sequence comes close to a ratio of code amounts, on the coding-unit basis, of data composing said second data sequence.
According such method and apparatus, it is possible to perform the recoding faithful to original rate control, and to suppress image quality deterioration due to errors accumulated by requantization, by recording the code amount of a frame of original image data, and setting the target value of the generated code amount of the second data sequence at the value obtained by multiplying the recorded code amount by the predetermined factor in transforming the data in the second moving picture coding system.
A third aspect of the present invention is a method and apparatus for transforming, sequentially from a head of data, a first data sequence generated by a first moving picture coding system that switches an intra-frame coding mode and inter-frame coding mode on a block-by-block basis to a second data sequence generated by a second moving picture coding system that switches the intra-frame coding mode and inter-frame coding mode on the block-by-block basis, and the method and apparatus select the intra-frame coding mode more in the second moving picture coding system than in the first moving picture coding system in the case where a generated code amount of the second data sequence does not reach a target value.
According to such method and apparatus, the capacity of an output buffer is adjusted by inserting the intra-frame coding mode when the capacity left in the output buffer is relatively large. It is thus possible to perform rate control without changing quantization characteristics, and to suppress image quality deterioration caused by requantization.
A fourth aspect of the present invention is a method and apparatus for transforming, sequentially from a head of data, a first data sequence generated by a first moving picture coding system that switches an intra-frame coding mode and inter-frame coding mode on a block-by-block basis to a second data sequence generated by a second moving picture coding system that switches the intra-frame coding mode and the inter-frame coding mode on the block-by-block basis, and the method and apparatus select the inter-frame coding mode more in the second moving picture coding system than in the first moving picture coding system in the case where a generated code amount of the second data sequence exceeds a target value.
According to such method and apparatus, the capacity of an output buffer is adjusted by inserting the inter-frame coding mode when the capacity left in the output buffer is relatively small. It is thus possible to perform rate control without changing quantization characteristics, and to suppress image quality deterioration caused by requantization.
A fifth aspect of the present invention is a method and apparatus for sequentially transforming a first data sequence generated by a first unlossless moving picture coding system with an information loss caused by quantization to a second data sequence caused by a second unlossless moving picture coding system with an data loss generated by quantization, and the method and apparatus:
1. record a quantization scale in reading data from the first data sequence; and
2. change a quantization scale to integral times the acquired quantization scale in adjusting the generated code amount of the second data sequence.
According to such method and apparatus, it is possible to suppress image quality deterioration caused by requantization, by changing the quantization scale by integral times when the rate control is performed by changing the quantization scale.